Benchmarking Self-Supervised Contrastive Learning Methods for Image-based Plant Phenotyping

Image-based plant phenotyping enables the high-throughput measurement of the physical characteristics of plants by combining one or more imaging technologies with image analysis tools. Over the past decade, deep learning has been widely successful for image-based tasks like image classification, object detection, image segmentation and object counting. While deep learning has been applied to image-based plant phenotyping tasks like plant species classification, plant disease detection, and leaf counting, its application has been limited. Part of the reason for this is that deep learning models tend to rely on large annotated datasets for training, and it can be expensive and time consuming to generate such datasets. Motivated by the need to leverage unlabelled data, a lot of research effort has recently been directed towards the area of self-supervised learning (SSL). The common theme among various SSL methods is that they derive the supervisory signal from the data itself, usually by distorting the input in some way and learning features that are invariant to the distortions. Despite the surge of research in this area, there has been a paucity of research applying self-supervised learning on image-based plant phenotyping tasks, particularly detection and counting tasks. We address this gap by benchmarking two self-supervised learning methods -- MoCo v2 and DenseCL -- on four image-based plant phenotyping tasks (the downstream tasks): wheat head detection, plant instance detection, wheat spikelet counting and leaf counting. We study the effects of the domain of the pre-training dataset on the transfer performance using four large-scale datasets: ImageNet (general purpose concepts), iNaturalist 2021 (natural world images), iNaturalist 2021 Plants (plant images) and the TerraByte Field Crop datatset (crop images). To understand the differences between the internal representations of the neural networks trained with the different methods, we applied a representation similarity analysis technique known as orthogonal Procrustes distance. Our results show that (1) Finetuning a model that is pre-trained with an SSL method typically outperforms training from scratch for a downstream task, (2) The Supervised pre-training method outperforms DenseCL and MoCo v2 for all the downstream tasks, except for the leaf counting task where DenseCL excels, (3) There is not much difference, both in the downstream performance and the internal representations, between MoCo v2 and DenseCL pre-trained models, (4) Pre-training with the iNaturalist 2021 Plants dataset leads to the best downstream performance more often than other datasets, and (5) Models pre-trained in a supervised manner learn more dissimilar features towards the last layers compared to models pre-trained with MoCo v2 or DenseCL. We hope that this benchmark/evaluation study will inspire further studies towards the development of better self-supervised representation learning methods for image-based plant phenotyping tasks

Real-Time Sensory Information for Remote Supervision of Autonomous Agricultural Machines

The concept of the driverless tractor has been discussed in the scientific literature for decades and several tractor manufacturers now have prototypes being field-tested. Although farmers will not be required to be physically present on these machines, it is envisioned that they will remain a part of the human-automation system. The overall efficiency and safety to be attained by autonomous agricultural machines (AAMs) will be correlated with the effectiveness of information sharing between the AAM and the farmer through what might be aptly called an automation interface. In this supervisory scenario, the farmer would be able to both receive status information and send instructions. In essence, supervisory control of an AAM is similar to the current scenario where farmers physically present on their machines obtain status information from displays integrated into the machine and from general sensory information that is available due to their proximity to the operating machine. Therefore, there is reason to expect that real-time sensory information would be valuable to the farmer when remotely supervising an AAM through an automation interface. This chapter will provide an overview of recent research that has been conducted on the role of real-time sensory information to the task of remotely supervising an AAM

Global Wheat Head Detection Challenges: Winning Models and Application for Head Counting

Data competitions have become a popular approach to crowdsource new data analysis methods for general and specialized data science problems. Data competitions have a rich history in plant phenotyping, and new outdoor field datasets have the potential to embrace solutions across research and commercial applications. We developed the Global Wheat Challenge as a generalization competition in 2020 and 2021 to find more robust solutions for wheat head detection using field images from different regions. We analyze the winning challenge solutions in terms of their robustness when applied to new datasets. We found that the design of the competition had an influence on the selection of winning solutions and provide recommendations for future competitions to encourage the selection of more robust solutions